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Abstract: Trajectories into the future of art are projected by investigating the basic 
question: How does computer art differ from traditional art? The author proposes 
that the computer as a symbol-processing machine is capable of simulating mental 
acts similar to the creative activity that is embodied in a work of traditional art. 
Therefore, the computer artist becomes empowered not only to craft a particular art 
object with the computer but rather to design another art subject in the form of a 
machine that is autonomously able to perform artistic activities and create art objects 
‘itself. 


I, PHYSICAL - SENSUAL - MENTAL 


"While in the process of making a work the artist might be said to ‘experiment’ with 
his own sensibilities" [1]. Twenty years ago, Frank Malina summarized his artistic 
credo in this statement. The astronautic scientist Malina transposed the principles of 
experiments into the sphere of art and maintained that they are valid operations in 
art as well as in science. Malina considered art and science equally as testing grounds 
for the unknown, differing only in their subject matters. Whereas he viewed science 
to be concerned with knowledge, for him art occupied the realm of the senses. 
Malina's interest was primarily to investigate kinetic art as a way of dynamically 
changing visually perceivable phenomena. In kinetic art, the physical arrangement 
of a specific contraption is moving and thus constantly prompts the senses of the 
viewer to adapt to altered perceptions of the object's states determined by its 
physicality even if it is as intangible as light. By contrast, in computer art the 
physical boundaries of the architecture of the machine are bypassed by the fact that it 
is a symbol processing system. 


The processing of symbols, understood here as formal tokens, constitutes the 
computer as a dynamic system of a very different type; what is being moved and 
changed by a digital computer is the particular organization of these binary tokens - 
elsewhere I define them‘as ‘data particles’ to emphasize their ontological status as 
the urform of informatiorial matter [2] - as well as their representational function. 
The dynamic and non-physical change of the symbols' syntactic and representational 
structure, has repercussions for their cognitive value. Let me then expand on 
Malina's notion by including ‘mentalities’ into his concept of experiments in the 
arts. After an apparently innocent transformation, his statement reads: While in the 
process of making a work the artist _— be said to ‘experiment’ with his or her 
own mentalities. 
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The computer's potential for symbol processing causes a significant reversal 
from the physical to the mental and vice versa by visualizing the structural state of 
the symbols. The act of transposing the internals of the machine into the sensual . 
domain of humans is called, in today's jargon, simulation, if it follows rules found 
in reality and derived out of the lawful relation between physical objects. On one 
hand, the repercussions of digital simulations are an ever-increasing 
dematerialization of, among others, aesthetic activities and objects. On the other 
hand, they provide sensuously apprehendable simulations in situations where 
scientists previously felt compelled to check theories with thought-experiments 
because the instrumentarium used to conduct and measure physical experiments 
was inadequate to provide measurable data. With Malina I acknowledge 
experimentation to be a fundamental method in science and in art. This includes 
the legitimate appropriation of any technology for artistic purposes and the 
application of suitable procedures to experiment with these technologies in order to 
find visually meaningful expressions. But, being a child of today's proliferation of 
computers in all scientific and social domains, I shall stress the meaning of 
experiment in terms of simulation of the unknown. 


Especially in the early times of quantum physics, thought-experiments were 
frequently used as a potent heuristic strategy whenever a new theory tried to explain 
results obtained with inadequate measuring instruments. "A thought-experiment is 
a mental exercise. It has the advantage of requiring no apparatus other than the 
mind, which frees it from the practical limitations of laboratory experiments" [3]. 
Thus, theoretical assumptions could be tested for their logical consistency. By the 
same token, and possibly more important, thought-experiments enabled physicists 
to use their imaginations freely as a methodical means to invent new hypotheses to 
guide future experimental designs. Imagination often was brought to bear in form 
of the visualization of properties that could not be grasped in reality but only 
contemplated in the mind [4, 5]. 


Digital simulation carries a similar heuristic principle: it makes sensuously 
apprehendable something that does not really exist. The kind of real, but simulated 
objects and their behavior as a computer simulation could be dubbed ‘aesthetic 
reality’, a term coined by the philosopher Max Bense [6]. Bense used this term to 
refer to works of Concrete Art and to identify their ontological being as aesthetical, 
i.e. they are made ‘concrete’ for and consumed by the senses, their physical reality 
exists only for this singular purpose. In conjunction with computer simulations, I 
use ‘aesthetic reality’ in the very literal sense of the words: simulations exist only as 
aesthetic objects and do not possess any concrete reality. In short, though 
simulations are not corporeal. nevertheless they are sensually existent. 


Writing this essay, I am conducting a thought-experiment in the sense that I 
am envisioning and mehtally testing my theories in regard to the question, How 
does computer art differ from traditional art? Challenged to write this article in the 
context of the future of art , 1 began to perceive that what was called for was a 
scrutinizing look at the very fundamentals of art, mind and machines. Only by 
revealing them, I think, can we attempt to venture beyond the surface criteria that 
seem to dominate much of the contemporary discussion about computer art. The 
‘look and feel' of computer generated imagery is less interesting to me than 
projecting extrapolation from known basic digital-symbolic functions into the future 
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potential of this new medium. Acknowledging the generative power of computers, | 
consciously limit myself here to only the productive aspect of computer art, even 
though I am aware that this art form also poses new questions in regard to the 
completion of the aesthetic communication cycle, particularily in terms of symbolic 
meaning and understanding by humans and machines. 5o please, follow me 
through this thought-experiment - designed as a 'guided tour’ rather than a 
‘Brownian walk' - with an open and critical mind. You are welcome to question my 
reasoning along the way, but you should be self-reflective and continuously examine 
the basis and consequences of your own opinions. If we leave this experiment with 
new insight into the potentials and limitations of computers in future art, we both 
will have benefitted. 


Our experiment will travel a route that takes us back to basic definitions of art, 
definitions that were drastically revised after the introduction of machines that 
could produce images. After a brief look at how some painters responded to this 
change by emphasizing the mental properties of the creative act, we turn to three 
prototypes of computer art to illustrate how programmable machines can be used to 
experiment with our mentalities. The trajectories from this discussion finally result 
in my suggestion that the digital artist is not limited to making art objects , but can 
create dynamic art subjects , machines that can themselves become autonomous 
devices capable of creating art. My suggestion is not unique - at least two pioneering 
computer artists, Robert Mallary and Hiroshi Kawano, anticipated similar ideas; both 
envision the computer as an independent ‘organism’ or as an ‘art-computer' that 
makes the art [7,8]. 


I]. ART OBJECTS AND THE THINKING CONSCIOUSNESS 


Obvious and inherent contradictions have marked the entire history of art 
because it has a ‘thingy’ nature and simultaneously speaks directly to human 
emotion and mind. Hegel tried to mediate this contradiction by distiguishing the art 
object from the natural object. Whereas nature simply is, the art object is and also 
exists for ourselves; thus art Serves as a means for the human consciousness to 
reduplicate itself. Art is then a method of mirroring the thinking consciousness by 
externalizing its content. And by becoming an external object that presents itself to 
the mind, art becomes the manifestation and origin of other mental activities. 
Therefore, the creation of art and its perception can be understood as a dialectical act 
of becoming conscious of ourselves. We are raising our own consciousness about 
ourselves to a new level by reflecting it in the form of an external object and by 
reflecting in turn on this object itself. 


In the following discussion, I quote comprehensively from Hegel's Asthetik 
since I am convinced that his dialectic between art object and thinking consciousness 
is useful for our thought-experiment, even if we cannot consider the Hegelian 
system in its entirety here. Hegel says: 


The universal and absolute need out of which art, on its formal side, 
arises, has its source in the fact that man is a thinking consciousness, i.e. 
that he draws out of himself, and makes explicit for himself, that which 
he is, and, generally, whatever is. The things of nature are only 
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immediate and single, but man as mind reduplicates himself, inasmuch 

as prima facie he is like the things of nature, but in the second place just 
as he really is for himself, perceives himself, has ideas of himself, thinks 

himself, and only thus is active self-realizedness [9]. [lemphasis, here and 
in all other quotes, in the original] 


He then goes on to say: 


The work of art then, of course, presents itself to sensuous apprehension. 
It is addressed to sensuous feeling, outer or inner, to sensuous perception 
and imagination, just as is the nature that surrounds us without, or our 
own sensitive nature within. Even a speech, for instance, may be 
addressed to sensuous imagination and feeling. Notwithstanding, the 
work of art is not only for the sensuous apprehension as sensuous object, 
but its position is of such a kind that as sensuous it is at the same time 
essentially addressed to the mind, that the mind is meant to be affected by 
it, and to find some sort of satisfaction in it. ... For the sensuous aspect 
of the work of art has a right to existence only in as far as it exists for 
man's mind, but not in as far as qua sensuous thing it has separate 
existence by itself [10]. 


The more the artwork, as a spatially extended object, is contemplated in its 
compositional unity, the more it loses its raw physical character. The more the 
formal, compositional unity of the artwork and the objects it represents are reflected 
upon, the more it consequently becomes a unified object in reflection, i.e. in the 
mind. Contemplation of the artwork 'dematerializes' the thingy, sensory element of 
the work, assimilating its unities of reflection into thinking. By thus reflectively 
bringing the reflexive and compositional unity of the artwork to its fulfillment, the 
artwork is allowed in the mind to open up or disclose multifarious strands of 
reflection in the process of the mind itself. Thus, by being assimilated into the 
mind's reflexivity, the art work evinces dimensions of pure thought which 
constitute its dialectical ‘subjectivation'. The reflective subjectivation of the art 
object is equivalent to its 'spiritualization’: that is, through its assimilation by 
thought, both reflections and feelings, previously viewed as being in the external 
work itself, are activated in thought, becoming one with it. 


I claim that Hegel's dialectic between art object and the thinking consciousness 
can be made productive for the contemplation of contemporary art as well as 
computer art, for the fundamental meaning of art has not changed, even if its topics 
and methods have undergone significant revolutions. A major change that art has 
undergone, in particular since the turn of this century, concerns the 
representational faculties used to model consciousness. From the ‘narrative’ 
methods (whether symbolical-allegorical, classical or romantic) known in Hegel's 
time that were built on top of illusionary depictions of the 'real' world, we have 
moved to physical-painterly methods that attempt directly to embody mental states 
without reference to meanings suggested by, but outside, the art object itself. But this 
change, as dramatic as it may seem, does not affect the fundamental purpose of art 
itself: to be the tool for self-reflection of the human consciousness. 
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The British philosopher Richard Wollheim attests to the predominant shift 
toward the physical in the work of art. 


For the mainstream of modern art, we can postulate a theory that 
emphasizes the material character of art, a theory according to which a 
work of art is importantly or significantly, and not just peripherally, a 
physical object. ... Within the concept of art under which most of the 
finest, certainly most of the boldest, works of our age have been made, the 
connotation of physicality moves to the fore [11]. | 


Wollheim’s notion of the physicality of the work of art acknowledges the rejection 
of representing some aspect of reality in an illusionary manner. The artist, rather, 
creates a new aesthetic reality with the work which embodies in it the mental 
properties of the artist. By foregrounding the physical object, we have not 
surrendered the dialectical claim that the art object represents human consciousness 
in order to speak to it and be reflected by it. It might be said that modern art models 
with precision consciousness by following one of two approaches. In the next section 
we will look more closely at the approach where the art object vanishes in 
comparison to the mental conceptualization of it. Here, I only want to take note of 
the other approach whose epitome might be earthworks which, in a very literal 
sense, use the soil of the earth as the most physical material imaginable. Robert 
Smithson, a pioneering artist of earthworks, discussed his intentions in an article 
that characterized his art projects as "a sedimentation of the mind [12]." His remark 
reminds us that even the most physical art works of today are conceived of as 
mental representations. So there is no contradiction between conceptualists and 
earth artists, at least in this sense; both try to model mental events but choose 
extremely different material in which to craft their thoughts. 


Dewey nicely expressed the interplay between matter and thought that 
constitutes in his eyes the creative act: 


The artist has his problems and thinks as he works. But his thought is 
more immediately embodied in the object. Because of the comparative 
remoteness of his end, the scientific worker operates with symbols, words 
and mathematical signs. The artist does his thinking in the very 
qualitative media he works in, and the terms lie so close to the object that 
they merge directly into it [13]. 


Interestingly enough, in contrast to artistic material, Dewey mentions symbols 
which he correlates with scientific operations. He could not foresee that the advent 
of the computer as an aesthetic engine is precisely rooted in the artist's capability of 
crafting imagery with symbolic ‘matter’ alone. This symbolizing ability of the 
digital machine is, in my mind, what distinguishes the computer from any other 
physical material and is at the same time the condition for the machine's potential 
to simulate mental activities and to externalize them, just as an artist does in 
making art. 
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Ill. BACKPAGES OF THE FUTURE 


The art of the twentieth century is distinguished by a continuous and two-fold 
struggle of artists to account for the emergence of visual media such as photography, 
film, television, etc. On the one hand, they attempt to consciously integrate these 
media and to use them alongside more traditional means. But on the other hand, 
artists feel challenged to reflect on and to redefine artistic and creative endeavors 
because new optico-chemical and electronic media are significantly changing what 
visual representations and the processes of their creation are all about. Many of 
those most sensitive to the ‘machine age’ [14] feel threatend by a tendency that 
Siegfried Giedion's feliticious phrase ‘mechanization takes command’ expresses [15]. 
Technological development in conjunction with the uncovering of the unconscious 
strongly influenced artists’ quest to go beyond the pictorial imitation of reality. It is 
symptomatic that Kandinsky inaugurated a wave of non-representational painting 
at the beginning of this century by probing into ‘the spiritual in art' [16]. 


Let us first consider, in a cursory and incomplete overview, how the Futurists - 
vocal champions of the machine age, admirers of the machine's power, energy and 
speed - responded to the machine's intrusion into the arts. Giacomo Balla, for 
instance, demanded: 


It is imperative therefore not to halt and contemplate the corpse of 
tradition, but to renew ourselves by creating art that no machine can 
imitate, that only the artistic Creative Genius can conceive [17]. 


Since machines such as photo and film cameras can create images, Balla advocated 
an art form that 'no machine can imitate’. Such art required in his view, the 
intervention of the ‘Creative Genius’. Clearly, Balla reserved mental capacities for 
the human species and stipulated special powers as emanating from the artist. 
Moreover, he emphasized that creativity was the main ingredient ensuring that 
human-made art remained superior to any machine-generated images. Implicitly, 
he rejected any form of trompe I'oeil painting, since this was precisely the type of 
visual representation machines could easily reproduce. The significance of this 
rejection needs to be seen in light of the fact that techniques to produce an 
illusionary naturalism were the grand achievements of Western painting and 
remained valid for several centuries. All of a sudden, what had been the hallmark of 
visual representation is retired in favor of the immaterial capacities of the artist; the 
Creative Genius becomes the last bastion against the onslaught of increased machine 
performance. Having established a position that supposedly guarantees human 
superiority, the Futurists readily embraced the additional power gained with 
machines that were firmly, so they believed, under their control. 


While Balla nonchalantly posited human superiority in the competition with 
the machine, Moholy-Nagy, the art experimenter per se, pragmatically practiced a 
division of labor similar to and incorporating industrial production. For _ 
Moholy-Nagy, the only determining factor for a work of art was the ‘inventive 
mental process’ involved in its genesis. For its actual production, any available 
method could be commanded. 
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In comparison with the inventive mental process of the genesis of the 
work, the manner - whether personal or by assignment of labor, whether 
manual or mechanical - is irrelevant [18]. 


Like Balla, Moholy-Nagy put a premium on the mental capacities of the artist; but 
whereas Balla poetically insisted on the ‘Creative Genius’, thus reclaiming some of 
the lost aura of the work of art for the artist, Moholy-Nagy plainly referred to 
‘inventive mental processes’, using a terminology that would be quite acceptable for 
today's cognitive scientists. Since the cognitive engagement of the artist is 
responsible for the genesis of the work, the execution of its realization, whether 
manual or mechanical, becomes irrelevant. This move enabled Moholy-Nagy to 
freely take advantage of new materials and production processes. He could even 
propose and practice, for the first time in the history of art, a division of labor that 
resembled the division of labor in modern industry, while keeping alive the special 
status of art. 


It should be noted that industrial production is quite different from the 


-apprentice's work within a manufactury so common in medieval times. Not only 


are machines directly employed, the intimate personal relationship between master 
and apprentice is replaced by an anonymous business deal between artist and 
machinist. A second important principle of art has been given up: the 

craftsmanship of the artist. Previously, craftmanship was the precondition of the 
artist's intimate and skilled handling of the material; only through this kind of 
manual expertise could the material be crafted into its expressive form. Now, this 
handiwork is considered even to be limiting in face of various industrial materials 
that can better be worked on by a new type of professional with industrial machines. 
The artist is divorced from materiality and becomes a professional inventor. 


More recently, the conceptual artist Sol LeWitt moved beyond Moholy-Nagy's 
understanding of what constitutes art by claiming that not only is the mental process 
more important than the physical execution of the work, it is the only condition 
necessary to actually create the work. The artistic idea alone is sufficient and needs 
no physicalembodiment. => 


In conceptual art the idea or concept is the most important aspect of the 
work. When an artist uses a conceptual form of art, it means that all of the 
planning and decisions are made beforehand and the execution is a 
performatory affair: The idea becomes a machine that makes art [19]. 


Once an idea is considered to be a legitimate work of art, all previous notions about 
the art object vanish. No longer does it consist of physical material; no longer has an 
artist or anybody to spend physical energy to manipulate and form the material; no 
longer has the beholder anything to hold onto. How the idea - the constituting 
element of the work of art - is communicated is secondary to its very existence as an 
idea. Actually, as insinuated by Lucy Lippard, the realization contaminates the purity 
of the original idea [20]. Once made for real, the art ‘work’ loses its lucidity to convey 
mental concepts. Here we see the radical ramifications of the ongoing trend toward 
the mental in art: The object itself dissolves into a transcendental being; what has 
been the goal in creating art has been completely replaced by the mental process that 
thinks the art. 
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This admittedly cursory flashback into the history of twentieth century art 
demonstrates that artists have consciously severed their ties to physical activity toa 
point where the result of artistic work does not necessarily need to be embodied in a 
physically existent art object. The shift from the physical to the mental has been 
prompted by the arrival of machines that can produce the physical portion of 
imagery; the artist is still privileged to create the imagination. 


Let me then finish this overview with a quote by the Russian painter Kasimir 
Malevich that reveals his utopian imagination. 


My research has led to the conclusion that Suprematism contains the idea 
of a new machine, in other words of a new organic engine without wheels, 
and without power derived from either steam or gasoline [21]. 


Malevich had recognized the very material of painting’as the basis for his aesthetic 
constructions. He ultimately reduced the image to a single geometric object, the 
square, painted in the most basic colors, all black or all white. His paintings Black 
Square andWhite on White are bold painterly metaphors to mark the closing 
chapter of former representational painting and to open it to the 'non-objective 
world of pure feeling’. In this speculative-theoretical statement, Malevich remained 
true to himself when he used the metaphor of an ‘organic engine’ to describe his 
dreams of a future painting device. It should be noted that Malevich conceived of 
this machine as quite different from known mechanical devices by calling it organic . 
Even if he was unable to further qualify his vision, it is clear that he had in mind 
something that could rival the human organism. His dream machine, too, was to 
overcome its mere physicality by not being powered by material energy such as 
steam or gasoline. 


Next, we will see how the advent of the digital engine pushes the concept of 
the artwork and the process of its creations into the non-objective, or more precisely, 
non-material world of symbols. 


IV. THREE CASE STUDIES 


In tune with my thesis of the ongoing dematerialization of the art object, I find 
the most interesting projects to be those that cast a shadow forward by experimenting 
mentally with the capabilitites of the computer. Such experiments do not fall into 
the mainstream of computer graphics which currently is occupied by attempts either 
to render superrealisti¢ representations, i.e. to simulate photorealistic 
representations, or to act as a substitute for traditional artistic media such as 
painting. It is obvious that, at least in the short run, such surrogates will gain 
widespread acceptance because of their economical feasibility, inviting accessibility 
and immediate practicality. Nevertheless, a few radically different approaches are 
being explored that promise to alter common standards of creation and perception. 
Prototypes of this kind will exert their impact in the future and will change the 
relationships between artist, art and audience more profoundly than the modest 
ambitions of the realist persuasion could ever predict. 
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Machine Vision: Looking Through Machine Eyes 


From the outset of working with the brand-new medium of video, Woody 
Vasulka, a Czechoslovakian-born film maker who now resids in the United States, 
was drawn to its material and basic qualities: the electrons that scan the image ina 
constant, regular pattern. Charged with different voltages, these electrons contain 
information about color intensities with which they strike the phosphor of the 
screen momentarily, only to be consumed a few nanoseconds later by the next 
electronic wave. Woody and his wife and collaborator, a former musician, Steina, 
soon became obsessed with time/energy as the material for electronic imaging; they 
started using, then devising and building, special machines to explicitly influence 
and control the waveforms, the ephemeral substance of the video image. Even 
though they continued to use the camera for capturing 'real' images that later could 
be modified and deformed, they also devoted many of their experiments to working 
exclusively with internally generated oscillations that, after various processing steps, 
appeared on the picture screen transformed into visual energy. Woody Vasulka 
recalls: "Our goal was to create reality, a certain reality that would testify to its own 
electronic complexities" [22]. 


Given this attitude that tries to do justice to the imaging properties of 
electronic machines, it is no surprise that Vasulka strongly advocates a systematic 
experimentation with all the parameters involved, without constraining them 
beforehand by our cultural conditioning of what constitutes a meaningful and thus 
legitimate image. He actually goes so far as to consciously challenge conventional 
image making by bypassing techniques that are based on the common perspective 
projection. He states quite rebelliously: 


I can at least unleash some attack against the tradition of imaging, which I 
see mostly as camera-obscura bound, or as pinhole-principle defined. This 
tradition has shaped our visual perception not only through the camera 
obscura, but it's been reinforced, especially through the cinema and 
through television. It's a dictatorship of the pinhole effect, as ironic and 
stupid as it sounds to call it that [23]. 


After having divorced himself from the highly limiting concept of 
photo-realism, Vasulka felt free to enter a period of experimentation with the 
Rutt-Etra video synthesizer, a device that uses intensity information to deflect the 
scanning beam of the cathode ray tube (CRT). In essence, in place of the 'empty’ 
regularity of the scanning rhythm, originally set up simply to move the beam across 
the screen plane, is now charged with image properties; this, of course, disturbs the 
scanning pattern and simultaneously deforms the video image itself. What we see 
is, in Johanna Gill's words, "a topographical map of the brightness of an image" [24]. 
[see figure 1] These are unfamiliar images indeed, images that no longer valorize the 
conventions of 'reality’; instead they allow novel perceptions of well-known 
objects, extract new meanings from them and press us to question the previously 
unquestionable, the validity of our own sensual-perceptual mechanisms. 


Once we have seen, that is, have sensuously experienced, alternative visions, 
our perceptual convictions can never remain innocent; by their mere existence, 
these machine visions expand the scope of how the world can be looked at, and 
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simultaneously they enrich the expressive means of artists such as the Vasulkas. 
For, once artists have encountered the spectrum of potential transformations which 
at first might come unsuspected and by surprise, they can catalogue them and make 
them part of their visual vocabulary. Woody Vasulka calls such results of their 
visual investigations ‘artifacts’. These visual signs are artifacts in that they could not 
have been premeditated by the artist and then simply realized; rather, these artifacts 
emerged from the artist's dialogue with the machine or, put even more strongly, 
they are manifestations of the machine's structural architecture as it dynamically 
processes visual information. The physicality of such artifacts is subsequently 
transmuted into symbols capable of triggering cognitive events by the reflective labor 
of the artist, a reflection that endows the purely formal utterances of the machine 
with meaning and makes sense of them. 


This is how Woody Vasulka judges the dialectical process between the 
machine and himself: "By artifacts, I mean that I have to share the creative process 
with the machine. It is responsible for too many elements in this work. These 
images come to you as they came to me - in a spirit of exploration" [25]. In another 
interview Vasulka considers the machine to be a device for the amplification of his 
fantasy: 


What intrigues me about computer and video are mostly the changes 
between time and other problems ... that cannot be foreseen or fantasized 
through the best fantasy synthesizer, which is the human brain. ... This 
untapped wealth is the pool of unmatched fantasy, fantasy that cannot be 
produced by plainly human fantasy, confined in a pictorial tradition [26]. 


Here we see an admission of human finiteness in conjunction with an 
understanding of a machine's unique contributions to image making. Note how 
much this statement differs from Balla's arrogant insistence on human superiority 
when he tried to confine a machine's capacities to rudimentary forms of imitation 
and anxiously reserved all potential for creation to the Creative Genius. Vasulka 
already approaches his interaction with the machine as a symbiosis in which no 
participant is privileged a priori. 


For the Vasulkas, it was a consistent development to start using digital 
computers once their price had dropped enough to make them accessible to 
individual artists and their power had increased sufficiently to make them suitable 
for video imaging. During the mid seventies, the Vasulkas with the help of the 
digital designer Jeff Schier, built an ‘image articulator’, a digital video machine that 
contained up to eight frames of image memory, analog-to-digital and 
digital-to-analog converters, and an arithmetic logic unit (ALU). The converters are 
important initially to digitize a holistic video image into discrete binary particles and 
then, at the end of the pipeline, to make the digital information accessible again to 
the human viewer in the form of a video image. The machine was optimized to 
operate in ‘real-time’, video-time that is, processing full video frames at 1/30th of a 
second. The Vasulkas had to trade off spatial and depth (color) resolution in order to 
achieve such high processing speeds, which they were quite willing to do, if only the 
pressing demands of the video-timing could be met. Out of the myriad of novel 
visual representations, that the Vasulkas were able to create with the image 
articulator, I will select only one example for discussion. 
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While experimenting with the arithmetic logical compositing of two images 
that were concurrently stored in two of the image buffers, Woody Vasulka came 
upon a strange phenomenon - logical motion. The initial set-up was as follows: 
Vasulka had two image sources. One came from a camera monitoring a simple 
object made out of white styrofoam, a ball. The other was an internally generated, 
abstract but well-structured computer image. The video image was digitized and the 
values of both images stored in a bitmap (4 bits deep for a total of 16 shades of grey or 
colors). Vasulka wanted to create a visual table that contained as entries all the 
combinatorial possibilities of compositing these two images, using Boolean algebra 
to combine their numeric values pixel-by-pixel. Some of these calculations yield 
visual results that are commonly used in television production. For instance, an 
addition is the equivalent of a superimposition of two image sources. The majority 
of Boolean operations (they include AND, NAND, OR, NOR, EXOR and inversions 
of the original images) result in composites that can be achieved only by these digital 
methods and have no analogue equivalent. This is the reason they have been 
non-existent before and are not charged with visual meaning, at least not yet. [see 

figure 2] 


The ‘logical motion’ effect occurred when Vasulka replaced the regular shape 
of the computer-generated image with a random pattern. He combined this image 
containing an arbitrary distribution of grey values with the video image of the ball. 
He fed the video image into the framebuffer not only once but continuously and 
calculated new values by merging the incoming image with the stored previous one. 
To his surprise, he saw a disc on the screen the pixels within the perimeter of the 
disk were percolating at high speeds, whereas outside the disc the pixels remained 
motionless. Vasulka was amazed that completely static images (the ball was not 
moving) could somehow create this motion. When he stopped feeding this static 
image of the ball into the computer, the motion vanished; all that remained visible 
was an unstructured, random picture. (Out of obvious reasons this effect cannot be 
shown in a static medium such as this journal. Please refer to Woody Vasulka's 
videotape ‘Artifacts’ (27].) 


After some reflection, he resolved the mystery: what he was seeing when the 
ball's image was continuously fed into the computer and composited with the 
random image was the difference of light intensities between the white ball and the 
black background. This difference had been translated into higher or lower binary 
numbers by the digitization process. Pixels within the disk were changing their 
values faster than those outside it because they represented the white ball's higher 
light intensity. The pixels representing the black background accumulated, on the 
average, higher values only gradually and, as a consequence, did not change as fast. 
Therefore, a high contrast was created between pixels in the two image areas, a 
contrast that depended.on the different rates of change rather than on noticeable 
differences in distinctly separate colored areas. Due to the random image, the entire 
image plane was unstructured; therefore, all that became visible was the ‘logical 
motion’ which immediately stopped once the feed with the ball's image was 
interrupted. 


The upshot of this experimentation is a remapping of visual information 
originally derived from a ‘realistic’ into a qualitatively different kind of 
representation. In this case, a representation of light intensities, our common way of 
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using optical means to project light rays onto a picture plane is interpreted as rate of 
change. And only this change is what remains visible, since any differences in 
intensity have been randomized throughout the process. As understood by Vasulka, 
we get to see the ‘collision of code’ that produces this remarkable motion. Since we 
have never seen digital code in action, it is no wonder that we initially have 
difficulties making sense of it. But I agree with the Vasulkas that once we have 
grown familiar with such visual structures, suggested and produced by the digital 
machine, we will start to utilize them to understand computers and to convey 
meaning among ourselves. The Vasulkas have been using techniques found in 

these experiments in their later video pieces by incorporating such images in an 
artistic compositional manner. We too will learn to see the world with a machine's 
eyes. 


Digital Dada: The Representation of Visual/Binary Symbols 


Hubert Hohn, originally a photographer, produced a mind-boggling piece 
entitled Binary Data Dump in 1984. (see figure 3) It consists of 256 sheets of 
computer printout, a floppy disc containing the print program and a list of assorted 
alternative titles for the piece. The printed image represents the state of the memory 
of an APPLE II personal computer that contains the program to produce this 
particular print-out. With this multifold assemblage, Hohn immediately presents a 
number of questions about the character of computer imaging and art to the viewer. 
The first questions concern the artwork itself: Where is it? Is it the printout, the 
floppy disk or the program on the floppy disk? It seems to me as if Hohn wants the 
entirety of different pieces replete with the long list of optional titles to be considered 
the artwork. By presenting it as this complex multi-media collage, he indicates that 
for him the computer and works produced with it encompass many different levels 
of signification that refer to each other in a circular, self-referential fashion. 


In an accompanying statement, Hohn describes Binary Data Dump as an 
attempt eos 
to locate ourselves within the nature of the digital processs and see what we 
become. The digital computer is a symbol processor. 256 discrete patterns of 
signals are available for definitions and manipulation as symbols. 
Everything here consists of symbols created through the transformation of 
symbols by yet other symbols. There are no original thingy sort of things. 
The territory consists entirely of maps which refer to no territory but to 
other maps. What I do is on floppy disks, and what the computer does is on 
paper printouts..My work exists only as binary magnetic code, and means 
nothing except during its execution, which is done by the computer. ... 
Multiple titles are used to encourage the viewer to be conscious of the work 
in a variety of ways, thus allowing it to exist in more than one state. My 
intent is to enliven the concepts while being fully consistent with the 
discipline of computing, for a world of multistate hardware and 
multireferent symbols is nothing if not a world of optional meanings [27]. 


_ If we look at the printouts at face value, we see that Hohn's program renders 
visible the invisible. He directly puts before our eyes the content of a memory of the 
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computer, information that we normally cannot see and, to be sure, do not want to 
see, because our interest is usually not in how the computer stores information but 
rather in the information itself. For instance, nobody would ordinarily be interested 
in the arrangement of binary numbers that represents this text internally in the 
word processor. But that is precisely what we are confronted with in Hohn's piece. 
With a conspicuous completeness, he presents the informational internals in front 
of us, as they are, in their basic form of binary values. His program prints out the 
‘data particles’, those last and only remnants of physicality that reside in the 
machine. By printing the values of these particles which actually are tiny 
electro-magnetic charges rather than the numbers 0 and 1 that we see, he arrests 
their fleeing ontological status. Finally someone captured that which is constantly 
changing, under steady revision and always switching from its current to its 
opposite value. In this regard, this is thus a traditional picture: the symbols in it refer 
to realities outside of it, and they do so by relying on conventions that rule the 
relationship between picture and depicted. 


But what is really on display? Hohn says this a ‘self-portrait’ made by the 

_ computer and jokingly claims that its 'self-consciousness’ is located in the memory 
locations made visible by the print program. Of course, he knows that this machine 
does not have any self-awareness of its actions; but literally speaking, he is correct, 
since the image we see is the code of the program that prints itself. Not only does 
this raise the question of machine consciousness but also the problem of the 
original. Is there a digital original, and if so, where is it? I submit there is a digital 
original, but it cannot be located as a physical entity. Neither the printout nor the 
floppy disk containing the program nor the program loaded in a particular computer 
can claim the status of an authentic, i.e. non-reproducible original. Not only can any 
of the above be exactly copied (this is one fundamental phenomenon of the data 
particle), the data particle itself is not authentic but only referential. Here we arrive 
at the heart of digital simulation where the transduction from the physical to the 
symbolic and vice versa takes place. And here is precisely the reason computers can 
simulate physical reality by processing a material which externalizes mental states: 
symbols. 


Let me then approach this riddle by first explaining where the digital original 
can be found. Only one answer is possible: it is the particular pattern of binary 
symbols. To use Nelson Goodman's term, the original is the notation describing a 
singular succession of symbolic events [28]. The unalterable definition of a chain of 
symbols is the original, whether it is a musical score or - and this is important - a 
binary description of a picture. (A binary description of a single image or a binary 
description of a prodyction rule of a picture are equivalent as notations.) But 
Goodman did not see.or did not consider that, with the existence of digital machines 
and digital symbols, images can be described, and therefore copied, with the help of a 
notational system. Moréover, the existence of a notational system for digital imagery 
is the basic reason that imiage making can be divorced from the physical conditions it 
was subjected to before. The physical task of placing visual marks can now be 
delegated to a machine if this machine is capable of performing the score, i.e. if it is 
capable of processing the symbols. At the same time, such notational symbols can 
not only be executed but generated by the digital computer. This creation of symbolic 
objects is, at least potentially, linked to cognitive capacities. In the next example we 
will meet such a machine that autonomously functions as an art-making system. 
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AARON: World Knowledge and the Logic of Image-Making 


AARON has been maturing over the years like a child growing up. AARON is 
the brainchild of the painter Harold Cohen, who in the midst of a successful career 
realized that painting was no longer challenging enough for him and, instead of 
continuing to paint himself, set out to teach a computer how to paint. The results of 
his efforts are embodied in AARON, a computer program that together with a 
suitable drawing device, such as a remotely controlled ‘turtle’ automatically 
produces endless variations of line drawings. Since its first inception at the 
beginning of the 1970s, AARON has undergone constant revision, becoming more 
and more sophisticated, and thus has grown along with the programming and 
image formalizing expertise of its creator, Harold Cohen. 


When Cohen started using computers for making art he asked himself the 
question, What is an image? In a lucid discussion in ari article with the same title 
[30], he revealed his reasoning and disclosed its results and the actions he took. This 
article was addressed to an audience of researchers in artificial intelligence, but later 
Cohen demonstrated that he can explain his concepts equally well to a group of 
children. The latter discussion is proof that the strategies brought to play within the 
computer program resemble fairly basic image-making ideas that can easily be taught 
to children in a sensuous manner. Maybe this is so since children might 
unconsciously be using similar strategies themselves and therefore are receptive to 
rationalizations that help them to clarify and make conscious their own drawing 
activities. 


With the development of AARON, Harold Cohen has tried to tackle what he 
considers "the only question which differentiates the computer fundamentally from 
other tools. It is not, what can you do with a program, but, what can the program 
do? For the artist, the essence of the computer is its autonomy" [31]. Such an 
approach completely rejects using a computer as a painting or modelling device 
because in both instances the artist has to provide, through suitable interfaces, the 
data and the commands on which the computer operates. Since I am not able within 
the scope of this article to discuss the particular qualities of human-machine 
interaction in detail, I will bypass Cohen's argument and will agree that it is indeed a 
challenge to program a computer in such a way as to make the machine an 
independent agent of art-making. Cohen's system AARON achieves certain qualities 
of this goal, but presently it is at a high price. AARON is not only autonomous in 
that it can generate without human intervention numerous different drawings in a 
rather perculiar (not to say, personal) style, it is also completely shut off from the 
surrounding world - AARON is deaf and blind, it cannot hear and cannot see. These 
are, if we compare it with the human artist, severe handicaps. AARON is not able to 
expand its knowledge, cannot check its internal structure against the outside, does 
not participate in life and cannot learn anything new unless it is spoonfed by Harold 
Cohen with another routine in the program that might enable it to do a new trick. 


Harold Cohen is obviously aware of these limitations and is nevertheless 
proud that AARON can do what it does and that it has been expanding its drawing 
capabilities significantly over the years. So what can AARON do well? AARON can 
draw images all by itself without any intervention. No two drawings are the same. 
Each drawing exhibits a well-ordered, non-random structure. AARON's drawings 


FD - thought-experiments 14 July 2, 1987 





are fairly organic in that they are not mathematically prescribed geometric functions. 
Taken together, these abilities in and of themselves demonstrate a certain 
accomplishment in the skill of drawing, and the pictures prove it. But there is more 
to it. AARON has gone through two distinctly different phases of drawing lessons. 
In the first period, AARON was taught the basic structure of images and image 
making. In other words, AARON has learned image-making strategies that comprise 
issues such as simple repetitions, the figure-ground problem, closure and various 
other related techniques. It also can find out where it currently is drawing, 
remember what it has done before and locate empty space on the picture plane. All 
of these techniques have been analyzed by Cohen, formalized into rules and are now 
embodied within the program. 


The program guides AARON's drawing behavior and controls the 
decision-making process during the course of a drawing event. But the program does 
not directly determine the outcome of a particular drawing as if it were driven by a 
notation. Such inevitable determination is given, for instance, in the case of the 
pianola, where the notation of a piece is engraved in the roll that mechanically 
controls the playing of the tune. Similarly, most computer programs are strictly 
deterministic by executing their inherent commands consecutively. They produce 
clearly predictable results which only change with different input. In Cohen's case 
we observe that AARON is a program capable of heuristic searches, ie. AARON 
constantly asks itself WHATIF questions and it is only the conditional response to 
them that effects the next drawing event. For instance, "IF you are about to bump 
into a figure and what you are doing is a closed shape, THEN find another path to 
close the shape"[32]. Or, "WHATIF I am going to go off the page? Well, THEN start 
circling; otherwise, keep going. WHATIF I'm circling, and I'm facing the direction I 
started from? THEN stop circling and head straight back; otherwise, keep circling” 
[33]. We see that AARON is not quite blind as I stated previously; AARON is able to 
orient itself within the little drawing world it is operating. Since this kind of 
feedback provides answers to the questions it is asking, AARON can construct its 
decision making around its drawing activities. It can be said that AARON contains 
rule-based heuristic image-making strategies that are enacted depending on the local 
situations encountered and the kind of inquiries made. 


This first generation of AARON already could serve as an ‘evocation 
generator’ by ‘sustaining an illusion’ of intended meaning. This illusion rests on 
the clever exploitation of AARON's use of 


cognitive primitives [that] echo the structure of visual experience, which 
gives the image its plausibility. ... Like its human counterpart, AARON 
succeeds in delineating a meaning-space for the viewer, and as in any 
normal transaction not totally prescribed by prior cultural arrangements, 
the viewer provides plausible meanings [34]. 


Having achieved an illusion of visual meaning produced by the computer, Cohen 
took the next step and incorporated world-knowledge into his program. The latest 
pictures produced by AARON illustrate this newly devised methodology. [see figures 
4 and 5] The pictures clearly show trees, plants and human figures. Moreover, these 
objects are organized within an apparently three-dimensional space, those in the 
foreground obscure others in the back, those farther away are smaller than those 
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closer to the viewer, etc. Cohen reasoned that 


the human cognitive apparatus develops in the context of the real world: so 
that in some sense cognition is the way it is because the world is the way it 
is. The result of considering this proposition is that, where the earlier 
AARON had been limited to knowledge of image-making strategies, the 
new AARON is more explicitly concerned with knowledge of the external 
world and the function of that knowledge in image-making [35]. 


Taken together, the two strategies - internal image-making logic and (some) 
knowledge of the external world - account for even more impressive drawings by 
Harold Cohen's program. AARON is proof that it is possible to embody a painter's 
skills in rules within a computer program, to formalize and materialize in a 
programming language the artist's image-making consciousness, and then to leave 
it up to the computer that executes this program to do the actual material work and 
draw the image. This achievement convincingly contradicts many people's 
understanding of the process of making art. Only by seriously considering the 
_ potential of this and similar experiments in computer art will we be able to make 
reliable predictions about future developments; clinging to opinions that are tainted 
by emotional judgments about the nature of art clearly will not do. 


V. FORECAST FOR TOMORROW 


One can appreciate the drama of digital machinery entering the world of art - a 
world that has always been highly charged with connotations of creativity, 
spirituality, and the like - by observing the fundamental premises that the French 
philosopher of perception, Merleau-Ponty, posed in the early 1960s. In his essay, ‘Eye 
and Mind’, he makes the following assumption that completely neglects any 
potential that digital machines might have for image making: "The painter ‘takes 
his body with him,’ says Valéry. Indeed we cannot imagine how a mind could paint. 
It is by lending his body to the world that the artist changes the world into paintings" 
[35]. If Merleau-Ponty could not "imagine how a mind could paint” at 
approximately the same time the first computers were used in the arts, we should 
not be surprised to still find disbelief and misunderstanding today, even in view of 
the fact that minds are able to do just that with the help of computers. 


By carefully evaluating such extreme antropocentrism as brought forth by 
Merleau-Ponty's philosophy, we can gain insight into those features that constitute 
human art. This will be most important since any attempt to generate machine art 
will be compared and measured against it. Crucial to Merleau-Ponty's 
understanding of vision and art is the synthesis which binds body and mind 
together. . 


Everything I see is in principle within my reach, at least within the reach of 
my sight, and is marked upon the map of the 'T can’. Each of the two maps 
is complete. The visible world and the world of my motor projects are each 
part of the same Being. This extraordinary overlapping, which we never 
think about sufficiently, forbids us to conceive of vision as an operation of 
thought that would set up before the mind a picture or representation of 
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the world, a world of immanence and ideality [37]. 


Like other phenomenologists before him, Merleau-Ponty wants to foreground 
the ‘lived experience’ of human beings, and this attempt, for him, entails moving 
"from thought of seeing to vision in act" [38]. He therefore criticizes Decartes's 
abstractions which made space an empty entity void of living beings but perfectly 
suited for geometric computations. (It is ironic that computer graphics technology 
has developed to a point where Decartes's laws of three-dimensional space are no 
longer simply computed by algorithmic software but are directly embedded in 
suitably designed hardware architectures, for instance, the so-called ‘Geometry 
Engines’. Descartes's abstractions are physically embodied by graphics machines. In 
this regard, geometry is being made 'real’ again.) But Merleau-Ponty's insistence on 
the symbiosis of the body's motor acts and the mind's thinking as necessary for a 
totality of Being seems to be correct in that it puts thought and the symbolic means 
of thinking on the solid grounds of primary experience as enacted and lived by the 
body. 


Only when such conjunction and grounding are provided are the signs used in 
interaction and in making art intelligible and meaningful. This lesson was learned 
the hard way in early, unsuccessful attempts in artificial intelligence (AT) to model 
cognition. These ambitious attempts failed mainly because symbols were processed 
as purely formal tokens. Searle's attack [39] against AI and his skepticism that it 
could ever create machines that are conscious rest precisely in the formidable 
difficulty of ‘materializing' the content of symbols in form of ‘lived experience’. 
Today's researchers are regrouping to resolve the problem of how to model in a 
machine a similar dialectic between mental and corporeal which is the cornerstone 
of human Being and human-made art. Even if this feat could be accomplished 
satisfactorily, another, almost more severe difficulty is already lurking at the 
horizon: how can a machine bypass its own physicality and inbred rule systems [40]? 
The real test of making art is creativity, the ability to overcome outgrown rules and 
to posit new ones; in short, creativity requires a mind that is free from the world, as 
Sartre has pointed out: "For a.consciousness to be able to imagine it must be able to 
escape from the world by its very nature, it must be able by its own efforts to 
withdraw from the world. In a word it must be free" [41]. 


Although the obstacle of creating creativity lies in front of us, let me conclude 
as if we had already succeeded in removing it. (We might succeed in the long run, 
just keep in mind that we have only a few decades’ worth of experience in computer 
simulation behind us. and still centuries ahead of us.) 


Previously, we had created art objects in which, by reflecting on them, we 
found echoes of ourselves. Now we are creating another subject, the Other that is 
not a mechanical contraption, such as in kinetic art, but a dynamic, autonomous 
entity capable of producing and understanding symbols - a machine capable of 
communication. This Other is really another subject which we cannot presume to 
be similar to us even though it can simulate a similarity that can make it 
indistinguishable from us. This Other manifests itself in a material physicality that 
is not of our flesh, and it possesses a mind that is not our mind. Still, this 
machine-Other will be, in a slight modification of Benjamin's words, 'the Computer 
as Producer' [42]. By building this machine-offspring, humans will have achieved a 


FD - thought-experiments 17 July 2, 1987 





marvel of artificial creation, and with this achievement the ultimate prerogative of 
humans, to be creators, will have been transferred onto a machine itself. Will this 
conclusion of our thought-experiment ever become reality? If it does, what will be 
the repercussions for a society of humans and machines? | 
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